Skip to content

feat(billing): settle at the tier the request's input length selects - #417

Merged
argszero merged 1 commit into
mainfrom
feat/catalog-tiered-price-settle
Oct 17, 2026
Merged

argszero merged 1 commit into
mainfrom
feat/catalog-tiered-price-settle

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

Models can now declare optional [[models.tiers]], and settlement bills a request at the price step its input length selects.

Step 1 (496a723) landed the field and its validation but left settlement untouched, so every request was billed at the first step. This step:

  • carries the steps into the database — models.tiers_json (schema v16 → v17), written by db::seed_models from config and read back by dao::get_model_price in the same query it already ran;
  • selects the step at settle time in billing::select_tier, on this request's input tokens (cache-miss + cache-hit prompt tokens), falling back to the model's own single price when there are no steps or the input is below the first from_tokens;
  • keeps peak-hour prices orthogonal: peak_* is a model-level trio and applies on top of the selected step.

from_tokens is an inclusive lower bound ([from_tokens, next)). Vendors that publish an open interval are expressed by adding 1 in the data: MiniMax's "input >512k" → 512001, xAI's and Google's "prompt exceeds 200k tokens" → 200001.

Tier data is filled only where the vendor publishes a number (checked against the pricing pages on 2026-10-16/17):

model long-context step
minimax-m3 >512k — 4.20 / 0.84 / 16.80 CNY
gemini-3.1-pro >200k — $4.00 / $0.40 / $18.00
grok-4.6, grok-4.7 >200k — $4.00 / $1.00 / $12.00

Rows deliberately left single-priced: OpenAI splits short/long context but does not publish the boundary token count; Anthropic's pricing page has no long-context step; qwen3.7-flash publishes input/output per step but no cache-hit price, so writing a step would drop its cache-hit price to 0 (under-charging). Each is noted as pending, with the reason, in the config header.

The admin model editor now clears tiers_json on write: it is a config-owned field, and a hand-edited base price would otherwise leave stale steps charging long prompts. Manual edits fall back to the single price, and the next start re-seeds the row from config.

Cache-write price (cache_write_input_per_m) stays declaration-only in this step, and the config comment now says so: no upstream response path yields a cache-write token count today, so wiring it into billing needs an extra usage parse.

Related Issue

References rant 2026-10-16T14:08:14.003832+08:00 (tiered price and cache-write price), step 2 — settlement. Step 1 (496a723) landed the expression and validation.

Changes

  • Feature/fix described
  • Config/data-structure changes synced to the example file (config/config.example.toml) and the database roster (docs/architecture.md)

Tests

  • cargo test all green — 579 passed / 0 failed (8 new)
  • cargo fmt --check passes
  • New unit tests added
  • cargo clippy --all-targets -- -D warnings and a non-test cargo build with -D warnings are clean

New legs: select_tier boundaries at from_tokens−1 / from_tokens / from_tokens+1 (and past the last step); a settle-level test that drives the real path (seed_models → dao → select_tier → ledger) and reads the recorded points back across the 200k boundary; a leg fixing the selection basis to cache-miss + cache-hit; and tiers_json round-trip plus malformed-JSON fallback legs. Both boundary legs were shown to bite by mutating the implementation (dropping the cache-hit term; making the lower bound exclusive) and watching them go red.

Checklist

  • Branch naming follows the convention (feat/)
  • Commit message uses Conventional Commits
  • Single responsibility, minimal change

Models can declare optional `[[models.tiers]]`: price steps chosen by the
input token count of a single request — cache-miss plus cache-hit prompt
tokens, not the user's cumulative usage and not the output. Step 1 landed
the field and its validation (496a723) but left settlement untouched, so
every request was billed at the first step. This step carries the steps
into the database (`models.tiers_json`, schema v17) and settles at the
selected one; a model without steps, or a request below the first
`from_tokens`, keeps billing at the model's own single price — byte for
byte as before.

`from_tokens` is an inclusive lower bound (`[from_tokens, next)`). Vendors
that publish an open interval are expressed by adding 1 in the data:
MiniMax's "input >512k" → 512001, xAI's and Google's "prompt exceeds
200k tokens" → 200001.

Tier data is filled only where the vendor publishes a number, verified
against the pricing pages: minimax-m3 (>512k), gemini-3.1-pro, grok-4.6
and grok-4.7 (>200k). OpenAI splits short/long context without publishing
the boundary, Anthropic's pricing page has no long-context step, and
qwen3.7-flash publishes input/output per step but no cache-hit price, so
those rows stay single-priced and are noted as pending in the config.

The admin model editor clears `tiers_json` on write: it is a config-owned
field, and a hand-edited base price would otherwise leave stale steps
charging long prompts, so manual edits fall back to the single price (the
next start re-seeds it from config).

Tested: cargo test 579/0, fmt/clippy/non-test build clean. Boundary legs
at from_tokens−1 / from_tokens / from_tokens+1, a settle-level leg that
reads the recorded points back across the 200k boundary, and a leg fixing
the selection to cache-miss + cache-hit.

rant 2026-10-16T14:08:14.003832+08:00 (tiered price and cache-write price)
@argszero
argszero merged commit 544cf86 into main Oct 17, 2026
2 checks passed
@argszero
argszero deleted the feat/catalog-tiered-price-settle branch October 17, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant